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1.   INTRODUCTION 
1.1   Background 

Computer  systems  are  playing  a  steadily  increasing 
role  in  every  aspect  of  society.   Education,  business,  and 
government  are  just  a  few  of  the  areas  where  people  find  it 
necessary  to  communicate  with  digital  computers,  and  the  lat- 
ter are  very  often  represented  by  interactive  terminals.   Sur- 
prisingly little  is  known,  however,  about  the  human  factors 
involved  in  communication  between  human  beings  and  computer 
terminals.   R.  S.  Nickerson  [8]  pointed  out  in  1969  that 
".  .  .there  is  remarkably  little  evidence  of  research  that 
has  been  undertaken  for  the  express  purpose  of.  .  .increasing 
our  understanding  of  man-computer  interaction.  ..."   Al- 
though some  research  has  been  carried  out  and  some  suggestions 
have  been  made  since  then  (see  Martin  [7]  for  examples) ,  there 
is  still  much  to  be  learned  in  the  realm  of  man-computer  com- 
munication. 

The  study  described  in  this  report  looks  into  one 
specific  aspect  of  man-computer  interaction--student  perform- 
ance at  on-line  terminals  under  varied  methods  of  error  inter- 
ruption.  Presently  there  are  quite  a  few  students  learning  to 
program  through  the  use  of  interactive  computer  systems,  and 
many  of  the  "interactive  compilers"  which  accept  student  pro- 
grams typed  at  a  terminal  have  one  feature  in  common:   when  a 


student  commits  an  error,  he  is  immediately  interrupted  by 
the  system  and  forced  to  correct  his  error  before  continuing. 
(See  Wilcox  [9]  for  full  description  of  such  a  system.) 

This  type  of  error  interruption  may  damage  the  stu- 
dent *s  train  of  thought  and  thus  adversely  affect  his  success 
at  the  terminal.   The  communication  process  from  human  being 
to  computer  terminal  can  be  looked  at  as  consisting  of  two 
parallel  processes — a  thought  process,  in  which  the  human 
generates  ideas  in  his  mind,  and  an  output  (typing)  process 
in  which  he  transfers  these  ideas  to  the  terminal.   A  student 
might  be  typing  one  idea  into  a  terminal,  while  preparing 
further  ideas  mentally.   At  this  point,  a  system  interruption 
could  disturb  the  balance  of  thinking  and  typing,  and  there- 
fore it  might  be  better  for  the  student  if  the  system  were  to 
wait  until  he  finished  a  task  or  subtask  before  announcing 
errors . 

In  addition,  since  human  personality  cannot  be  di- 
vorced from  any  form  of  man-machine  interaction,  it  is  pos- 
sible that  personality  traits  are  an  important  factor  in  the 
effect  method  of  error  interruption  has  on  student  perform- 
ance.  Certain  types  of  people  may  benefit  from  a  specific 
type  of  error  interruption.   (Consider  a  nervous  individual 
being  upset  by  immediate  system  interruption — or  would  he  be 
reassured  by  it?)   The  study  described  here  explores  some  of 
these  problems. 


1.2   Overview  of  Study 

Two  groups  of  students  (equally  matched  in  typing 
skill  and  experience  on  the  interactive  system  used)  were 
each  given  two  simple  tasks  to  perform  at  the  terminal.   Sub- 
jects in  one  group,  upon  making  an  error,  were  interrupted 
immediately  by  the  system  and  forced  to  make  corrections  be- 
fore going  on.   Subjects  in  the  other  group  were  allowed  to 
finish  each  task  without  interruption  until  they  signalled 
completion  of  a  task,  at  which  time  the  system  announced  errors 
and  required  corrections. 

The  tasks  used  were  simple  word  listing  problems: 

1.  List  25  of  the  50  states. 

2.  List  20  permutations  of  ' abcde *  such  that  the 
'c'  occurs  somewhere  before  the  'a1  and  the  *b' 
occurs  somewhere  before  the  'd'. 

Numerous  data  were  collected  on  subject  performance 
at  the  terminal,  and  a  standard  personality  test  was  adminis- 
tered to  the  subjects  to  obtain  data  on  basic  personality 
traits. 

The  following  experimental  questions  were  considered: 
Does  method  of  error  interruption  affect  subject 
performance  at  an  interactive  terminal,  and  if 
so,  how  do  these  effects  show  up? 
What  relationships  do  personality  traits  have  to 
performance  under  different  error  interruption 
methods? 


1.3  Limitations  of  Study 

The  study  described  has  several  limitations.   For 
example,  it  does  not  deal  directly  with  the  programming  task. 
Such  tasks  were  avoided  for  several  reasons.   First,  the  need 
to  teach  the  subjects  a  programming  language  or  ascertain 
their  level  of  skill  in  a  known  language  would  have  made  un- 
reasonable time  demands  on  the  study.   Second,  programming 
tasks  would  introduce  a  large  number  of  human  variables  which 
would  be  difficult  to  control,  since  so  little  is  known  about 
the  way  human  beings  tackle  programming  problems.   The  man- 
terminal  communication  process  is  the  focus  of  this  experiment, 
and  such  communication  goes  on  regardless  of  the  task  being 
undertaken. 

An  additional  limitation  is  the  fact  that  this  study 
does  not  examine  the  learning  process  of  subjects  studying  at 
computer  terminals.   The  question  of  which  method  of  error 
interruption  provides  a  better  learning  environment  for  the 
student  is  certainly  an  important  one.   However,  an  examina- 
tion of  this  question  would  require  a  much  larger  and  longer 
study  (for  instance,  a  semester  long  study) .   Also,  the  learn- 
ing process,  like  programming,  would  introduce  many  additional 
extraneous  variables  requiring  careful  control. 

1.4  General  Goals 

In  addition  to  examining  the  research  questions 


presented  in  Section  1.2,  it  is  important  to  find  out  whether 
we  can,  in  fact,  conduct  sensible  and  useful  experiments  in 
man-computer  interaction.   This  study  presents  examples  of 
what  factors  may  be  considered  and  what  measures  may  be  used 
in  evaluating  human  performance  at  interactive  computer  ter- 
minals.  Hopefully,  it  will  serve  as  the  basis  for  further 
research. 


2.   METHOD  AND  PROCEDURES 

2.1  Selection  of  Subjects 

The  population  under  study  consists  of  undergraduate 
students  learning  to  program  computers.   Seventy  subjects  were 
chosen  from  students  enrolled  in  100  level  computer  science 
courses  at  the  University  of  Illinois.   Approximately  one- 
half  of  the  subjects  were  members  of  one  course,  which  was 
run  through  the  experiment  during  a  scheduled  class  hour.   The 
rest  were  volunteers  from  other  basic  courses.   In  this  manner 
a  suitable  random  sample  was  achieved. 

To  prevent  bias  toward  one  error  interruption  method 
or  another,  subjects  were  not  told  the  exact  nature  of  the 
experiment.   They  were  told  instead  that  they  would  partici- 
pate in  a  study  of  common  sense  memory  at  computer  terminals, 
which  would  require  completion  of  several  simple  tasks.   After 
finishing  the  experiment  subjects  were  informed  of  its  actual 
purpose. 

2.2  Equipment 

The  computer  system  used  in  the  experiment  was  the 
PLATO  IV  system  [1] f  developed  at  the  University  of  Illinois. 
PLATO  (Programmed  Logic  for  Automatic  Teaching  Operations)  is 
a  computer-aided  instruction  system  consisting  of  a  CDC 
CYBER-7  3  mainframe  supporting  sophisticated  graphics  terminals 


with  plasma  screens.   This  system  is  used  in  a  variety  of 
departments  at  this  university,  including  the  Department  of 
Computer  Science. 

An  interactive  program  written  for  PLATO  administered 
the  terminal  session  of  the  experiment.   The  program  collected 
background  information  on  the  subjects,  assigned  them  to 
treatment  groups,  presented  the  tasks  to  them,  and  collected 
performance  data.   The  error-checking  algorithm  of  the  program 
(also  called  a  "lesson")  is  described  briefly: 

The  program  kept  an  array  of  correct  answers  to  the 
tasks,  and  in  addition,  a  pointer  table  containing  values 
such  that  table  (i,j)  pointed  to  the  first  answer  in  task  j 
beginning  with  the  i    letter  of  the  alphabet.   (The  value  of 
table  (27,j)  was  one  more  than  the  location  of  the  last  answer 
on  task  j.)   When  a  subject  began  typing  a  response,  the  point- 
er table  was  used  to  determine  the  region  in  the  answer  array 
where  the  response  might  reside.   As  each  character  was  typed, 
the  bounds  of  the  current  "correct  response  region"  were 
placed  on  a  stack.   When  the  subject  erased  a  character  the 
stack  was  popped  so  that  the  bounds  as  of  the  previous  charac- 
ter became  the  new  current  bounds.   These  structures  plus  a 
counter  for  number  of  characters  typed  so  far  allowed  for  ac- 
curate error  checking  in  each  response.   For  an  Immediate 
group  error,  the  lesson  responded  immediately,  while  for  the 
End  group  the  response  number  and  position  of  the  erroneous 
character  were  saved  for  later  announcement. 


In  addition,  the  Sixteen  Personality  Factors  Ques- 
tionnaire (16PF) ,  published  and  distributed  by  the  Institute 
for  Personality  and  Ability  Testing,  of  Champaign,  Illinois, 
was  used  to  collect  data  on  personality  traits.   The  test, 
developed  at  the  University  of  Illinois,  measures  sixteen 
basic  personality  traits.   These  are  listed  in  Table  2.1. 
Form  C  of  the  test  was  used  in  this  study. 

2.3   Presentation 

The  experimental  procedure  consisted  of  two  phases. 
The  subjects  were  first  given  the  16PF  Test,  which  took  approx- 
imately one-half  hour.   Following  this  they  proceeded  to  a 
PLATO  classroom,  where  the  terminal  session  took  place.   A 
PLATO  lesson   taking  about  a  half  hour  presented  the  actual 
experiment.   The  session  had  four  steps: 
a.   Background  Information 

The  lesson  began  by  collecting  the  following 
background  information  for  each  subject: 
Major  — 

1  -  Computer  Science 

2  -  Other 
PLATO  Experience 

0  -  Has  never  taken  or  written  a  lesson 

1  -  Has  taken  or  written  a  lesson 


Table  2.1 
Primary  Source  Traits  Covered  by  the  16PF  Test 


Factor 


Low  Score 


High  Score 


B 
C 

E 


H 


M 


N 


0 


Reserved ,  detached, 
critical,  aloof,  stiff 

Low  intelligence 

Affected  by  feelings, 
emotionally  less  stable, 
easily  upset,  changeable 

Humb 1 e ,  mild,  easily 
led,  docile,  accom- 
modating 

Sober ,  taciturn, 
serious 

Expedient,  disregards 
rules 

Shy,  timid,  threat- 
sensitive 

Tough-minded ,  self- 
reliant,  realistic 

Trusting ,  accepting 
conditions 

Practical ,  "down-to- 
earth"  concerns 

Forthright,  unpreten- 
tious, genuine  but 
socially  clumsy 

Self-assured ,  placid, 
secure,  complacent, 
serene 

Conservative,  respect- 
ing traditional  ideas 


Outgoing ,  warmhearted, 
easygoing,  participating 

High  Intelligence 

Emotionally  stable, 
mature,  faces  reality, 
calm 

Assertive,  aggressive, 
competitive,  stubborn 


Happy-go-lucky ,  enthusia- 
stic 

Conscientious ,  persistent, 
moralistic,  staid 

Venturesome ,  uninhibited, 
socially  bold 

Tender-minded ,  sensitive, 
clinging,  overprotected 

Suspicious ,  hard  to  fool 


Imaginative ,  bohemian , 
absent-minded 

Astute,  polished,  socially 
aware,  shrewd 


Apprehensive ,  self-re- 
proaching, insecure,  worry- 
ing, troubled 

Experimenting ,  liberal, 
free-thinking 
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Table  2.1  (continued) 


Factor 


Low  Score 


High  Score 


Q. 


Group  dependent ,  a 
"joiner"  and  sound 
follower 


Self-sufficient,  resource- 
ful, prefers  own  decisions 


Q, 


Undisciplined  self- 
conf lict,  lax,  follows 
own  urges,  careless  of 
social  rules 

Relaxed,  tranquil, 
torpid,  unf rustrated, 
composed 


Controlled,  exacting  will 
power,  socially  precise, 
compulsive,  following 
self-image 

Tense,  frustrated,  driven, 
overwrought 


Table  was  taken  from  the  manual  for  the  16PF  test  [6] . 
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Number  of  Years  Programming 

Number  of  years  (rounded  to  the  nearest 

integer)  subject  has  programmed 
On-Line  Interactive  Systems  Experience 

1  -  Has  previously  typed  a  program  into  an 

interactive  terminal 

2  -  Has  never  typed  a  program  into  an  inter- 

active terminal 

b.  Typing  Test 

Next,  the  lesson  administered  a  short  typing 
test  to  the  subjects.   Ten  fairly  large  words  were 
displayed  on  the  screen  and  subjects  were  asked  to 
type  the  words  as  quickly  and  accurately  as  possible 
(see  Figure  2.1).   The  total  time  elapsed  (in  tenths 
of  seconds)  and  the  total  number  of  keypresses  in 
error  were  collected  for  each  subject. 

c.  Group  Assignment  and  Randomization  of  Task  Order 

In  order  to  control  for  effects  of  typing  skill 
and  previous  experience  on  PLATO,  assignment  to  method 
of  error  interruption  groups  and  order  of  task  presen- 
tation were  determined  on  the  basis  of  these  two 
variables.   Four  types  of  subjects  were  defined: 

No  PLATO  Experience — Bad  Typist 

No  PLATO  Experience — Good  Typist 
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PLATO  Experience — Bad  Typist 

PLATO  Experience — Good  Typist 
Based  on  these  types  subjects  were  assigned  to  method 
of  error  interruption  such  that  each  method  had 
equal,  or  nearly  equal  numbers  of  each  subject  type. 
Order  of  task  presentation  was  randomly  assigned 
across  error  interruption  methods  and  subject  types 
(see  Table  2.2) . 

Typing  skill  was  computed  as  follows:   skill 
level  was  computed  as 

time  +  F  *  number  of  error  keypresses. 

The  scale  factor,  F,  was  computed  from  a  pilot  study 

(using  subjects  from  the  same  population  as  the  main 

study)  as 

mean  typing  test  time  (over  both  groups) 
mean  typing  test  errors  (over  both  groups) 

Skill  level  was  therefore  lower  for  subjects  with 

more  skill.   Main  study  subjects  were  ranked  as  good 

or  bad  typists  depending  upon  whether  their  skill 

level  was  less  or  greater  than  the  pilot  median 

value. 

From  typing  rank  and  PLATO  experience  (collected 

as  background  information)  subjects  were  classified 

into  the  four  types  mentioned  above. 
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Table  2.2 
Group  and  Task  Order  Distribution  by  Subject  Type 


Subject 

No.  of 

Type 

Member 

No  PLATO — 

Bad  Typist 

9 

No  PLATO — 

Good  Typist 

9 

PLATO — 

Bad  Typist 

19 

PLATO — 

Good  Typist 

33 

Group  Task  Order  * 

u     End        1-2         2-1 


10        9  8  11 

16       17         17  16 

Total  35      35        36         34 


Table  shows  breakdown  of  subject  types  within  error  inter- 
ruption groups  and  orders  of  task  presentation.   This  was  the 
distribution  arrived  at  by  the  PLATO  lesson  during  the  running 
of  the  experiment. 


Note:   Task  ordering  in  the  first  and  third  types  was  not  as 
good  as  it  could  have  been  because  several  subjects 
did  not  finish  the  experiment  and  the  incomplete 
data  produced  was  omitted  from  the  study. 


* 

1  -  States  task 

2  -  Permutations  task 


15 


d.   The  Tasks 

Once  error  interruption  method  and  task  order 
were  determined,  the  subjects  were  ready  to  work  on 
the  tasks,  which  were: 

1  -  List  25  of  the  50  states. 

2  -  List  20  permutations  of  'abcde1  such  that 

the  ' c'  occurs  somewhere  before  the  'a' 
and  the  'b'  occurs  somewhere  before  the  * d1. 
Previous  to  the  presentation  of  each  task,  subjects 
were  shown  full  instructions  on  what  was  required. 
Sample  screen  layouts  for  the  tasks  are  shown  in 
Figures  2.2-a,  2.2-b,  and  2.2-c. 

These  figures  also  show  the  two  methods  of  error 
interruption.   Figure  2.2-a,  the  Permutations  task, 
shows  Immediate  error  interruption.   Here,  at  the 
occurrence  of  an  error  keypress  a  box  was  drawn  around 
the  erroneous  character  and  a  message  given,  demanding 
that  the  error  be  corrected.   At  this  point  only  the 
-ERASE-  key  was  active,  and  the  subject  had  to  press 
this  key  and  then  retype  the  character  in  error. 

Figure  2.2-b  shows  the  States  task  with  End 
error  interruption.   In  this  case,  the  subject  has 
made  several  errors,  but  the  system  has  not  inter- 
rupted him.   After  each  response,  End  interruption 
subjects  were  allowed  to  go  back  and  retype  any 
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previous  responses.   This  option  was  also  available 
after  all  the  responses  were  typed.   Having  signalled 
to  PLATO  that  he  had  completed  the  task,  the  End 
interruption  subject  was  then  informed  which  of  his 
responses  were  in  error,  and  shown  the  first  erron- 
eous character  in  each  erroneous  response.   (This 
was  an  attempt  to  even  up  the  End  error  announcement 
facility  with  that  of  the  Immediate  group,  for  whom 
the  erroneous  character  was  boxed.)   The  lesson  then 
required  the  subject  to  retype  each  incorrect  re- 
sponse, as  shown  in  Figure  2.2-c. 

2.4   Data  Collected 

During  execution  of  the  tasks  the  PLATO  lesson  collected 
a  large  amount  of  data  on  subject  performance.   A  complete 
list  of  data  collected  is  given  in  Appendix  A.   The  list  below 
contains  only  those  items  used  to  calculate  the  dependent 
variables  described  as  performance  measures  in  Section  3.1. 
Each  item  given  below  was  collected  for  each  subject  on  each 
task. 

Total  Error  Keypresses — 

Total  number  of  erroneous  keypresses  during 
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* 

completion  of  a  task.    For  Immediate  interruption  this 

is  by  definition  the  number  of  error  interruptions  by 
PLATO.   For  End  interruption  this  is  the  number  of  error 
presses  which  were  not  preceded  by  uncorrected  error 
presses  in  the  same  response.   In  other  words,  for  the 
latter  group,  error  checking  in  a  response  was  suspended 
whenever  an  error  press  occurred,  and  reactivated  if  the 
error  was  corrected  by  the  subject.   This  was  done  be- 
cause of  the  difficulty  in  defining  errors  in  an  already 
erroneous  response. 
Total  Time — 

Time  elapsed  (in  tenths  of  seconds)  from  presenta- 
tion of  the  task  layout  page  to  successful  completion  of 
the  task. 
Total  Number  of  Keypresses — 

Total  number  of  keypresses  from  beginning  to  end 
of  a  task. 
Final  Answers — 

The  final,  corrected  set  of  answers  arrived  at  by 
each  subject  on  each  task. 
Error  Bit  Array — 

A  flag  array  denoting  each  response  in  which  an 
error  was  made. 


* 

Does  not  include  special  function  keys  such  as  super- 
script or  subscript,  which  were  inactive  during  the  experiment, 
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3.   RESULTS  AND  DISCUSSION 

The  aim  of  the  study  was  to  explore  the  following 
two  basic  questions: 

1.  Does  method  of  error  interruption  affect  stu- 
dent performance  at  interactive  terminals,  and 
if  so,  how  do   these  effects  show  up? 

2.  What  role,  if  any,  do  personality  traits  play 
in  student  performance  under  different  error 
interruption  methods? 

The  following  sections  describe  statistical  analyses 
conducted  and  results  obtained. 

3.1   Statistical  Analysis 

The  two  factors  considered  in  the  experiment  are 
given  below  with  their  respective  treatment  levels: 
Factor  A--Method  of  Error  Interruption 

Level  1 — Immediate  interruption 

Level  2 — End  interruption 
Factor  B — Task 

Level  1 — States  task 

Level  2 — Permutations  task 
Levels  of  each  factor  are  discussed  in  detail  in 
Section  2.3. 

Two  distinct  groups  of  subjects  were  used,  each 
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being  presented  with  a  different  method  of  error  interrup- 
tion.  Each  subject  completed  both  tasks,  however,  regardless 
of  group.   This  led  to  a  Two  Factor  Split  Plot  Design  (also 
called  Repeated  Measures  Design) .   In  this  design,  levels  of 
one  factor  (error  interruption  method,  in  this  case)  are 
divided  among  subject  groups,  while  all  levels  of  the  other 
factor  (task)  are  presented  to  subjects  in  each  group.   Fig- 
ure 3.1  shows  the  design  for  this  experiment.   (The  split 
plot  design  is  fully  discussed  in  Kirk  [5].) 

Subject  groups  were  matched  on  the  basis  of  PLATO 
experience  and  typing  skill,  and  the  order  in  which  the  tasks 
were  presented  was  randomized.   These  procedures  are  described 
in  Section  2.3. 

Two  reasons  explain  the  use  of  separate  subject 
groups  for  Immediate  and  End  interruption.   First,  presenta- 
tion of  both  methods  of  error  interruption  to  one  group  might 
have  introduced  bias  due  to  transfer  effects  from  one  method 
to  the  other.   Second,  more  tasks  would  have  been  necessary 
and  thus  the  experiment  would  have  become  awkward  in  length. 

In  order  to  examine  subject  performance  five  depen- 
dent variables  were  used.   These  performance  measures  were 
derived  from  raw  data  collected  during  the  experiment.   Each 
subject  produced  two  scores  (one  per  task)  on  each  measure. 
The  performance  measures  are  given  below  with  computational 
definitions    (see  Section  2.4  for  raw  data  relevant  to  these 
variables) : 
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Method  of 

Error 
Interruption 


States 


Task 


Permutations 


Immediate 


End 


gx(35) 

g-L(35) 

g2(35) 

g2(35) 

g.  and  g2  represent  the  two  subject  groups; 
group  size  is  given  in  parentheses. 


Figure  3.1   Two  Factor  Split  Plot  Design 
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Percent  Error  Keypresses 

Percentage  of  error  keypresses  on  the  task.   This 
was  calculated  as 

100  *  total  keypresses  in  error 
total  number  of  keypresses 

Total  Time  (seconds) 

Total  time  for  task  completion  converted  to  seconds 
Two  Consecutive  Responses  in  Error 

Number  of  occurrences  of  two  consecutive  responses 
in  which  the  subject  made  an  error. 
Number  of  Responses  in  Error 

Number  of  responses  in  which  subject  committed 
an  error. 
Keypress  Efficiency  (percent) 

Defined  as 


100  * 


number  of  keypresses  needed 
for  error  free  solution 


total  number  of  keypresses 

The  numerator  of  the  fraction  was  calculated  from  the 
final  corrected  set  of  answers  collected  for  each  subject 
on  each  task.   It  represents  a  "perfect"  solution,  i.e., 
the  number  of  keypresses  necessary  to  type  the  final 
answer  set  without  making  any  errors.   The  denominator 
of  the  fraction  was  collected  with  raw  data. 
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Numerous  other  variables  were  derived  from  the  data 
collected  during  the  experiment,  but  these  five  were  chosen 
as  the  most  reliable  performance  measures.   A  complete  list 
of  derived  measures  is  given  in  Appendix  B. 

The  various  statistical  analyses  conducted  on  these 
measures  are  described  next.   Some  of  these  analyses  were 
vital,  while  others  were  used  either  to  check  other  results 
or  to  insure  completeness.   The  analyses  are  described  below: 
T-Tests — 

Group  means  were  compared  on  typing  test  time  and 
errors,  and  on  number  of  years  of  programming  to  check 
that  these  variables  were  randomly  distributed  in  the 
two  error  interruption  groups. 

One-Way  Univariate  Analysis  of  Variance  (ANOVA) — 

This  simplest  type  of  analysis  of  variance  was 
conducted  across  method  of  error  interruption  for  each 
performance  measure  on  each  task.   Thus,  ten  such  anal- 
yses were  done.   Results  show  whether  error  interruption 
method  affects  any  of  the  performance  measures  consid- 
ering each  task  separately. 

Two-Way  Univariate  ANOVA — 

This  analysis  was  used  to  examine  the  combined  ef- 
fects of  error  interruption  method  and  task  on  each  of 
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the  five  performance  measures  taken  alone.   Five  two- 
way  analyses  were  done,  mainly  to  check  for  interac- 
tions between  the  two  factors. 

Multivariate  ANOVA — 

Multivariate  analysis  was  used  to  examine  the  ef- 
fects of  method  of  error  interruption  on  all  five  per- 
formance measures  taken  together.   One  such  analysis  was 
conducted  for  each  task.   In  multivariate  ANOVA  a  dis- 
criminant function  is  determined  such  that  its  applica- 
tion to  the  dependent  scores  results  in  maximum  group 
differentiation.   The  coefficients  of  this  function, 
called  discriminant  function  weights,  indicate  the  de- 
pendent variables  which  separate  the  groups  most  effec- 
tively.  These  analyses  serve  as  a  check  on  univariate 
results  in  that  a  significant  multivariate  result  in- 
sures that  univariate  effects  are  not  washed  away  when 
the  dependent  variables  are  considered  together.   In 
addition,  multivariate  ANOVA  determines  whether  the 
levels  of  a  factor  affect  certain  combinations  of  the 
dependent  variables.   (More  information  on  this  type  of 
analyses  can  be  found  in  Finn  [4].) 

To  examine  relationships  between  personality  and 
performance  a  two-way  multivariate  ANOVA  was  conducted 
on  the  personality  data  collected  by  the  16  PF  Test. 
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Subject  performance  was  estimated  as  a  function  of  total 
time  and  total  errors  on  each  task  and  was  sorted  into 
four  categories,  ranging  from  excellent  to  poor.   (See 
Appendix  G  for  details.)   Method  of  error  interruption 
and  performance  category  were  then  used  as  factors  in 
the  analysis,  with  the  sixteen  personality  scores  on 
each  subject  as  dependent  variables. 

Correlations — 

To  gain  additional  insight  into  personality- 
performance  relationships,  Pearson  product-moment  corre- 
lations of  the  performance  measures  with  the  personality 
scores  were  computed.   In  these  correlations  and  in  the 
multivariate  analysis  on  personality  data,  raw  scores 
from  the  16  PF  Test  were  used  rather  than  scores  stan- 
dardized by  norms  produced  by  the  general  population. 

3.2   Results 

The  T-Tests  for  differences  in  the  group  means  of  typing 
test  time,  typing  test  errors,  and  number  of  years  of  program- 
ming are  summarized  in  Appendix  C.   None  of  these  tests 
yielded  significant  results,  so  it  can  be  assumed  that  typing 
skill  and  programming  experience  were  randomly  distributed 
across  error  interruption  groups.   The  tests  were  done  as  a 
check  on  experimental  distribution  and  their  lack  of  signi- 
ficance lends  credibility  to  further  results. 


28 


Group  means  and  standard  deviations  for  each  per- 
formance measure  are  shown  in  Table  3.2-a  (States)  and  3.2-b 
(Permutations) .   (Summary  data  for  all  measures  computed  is 
given  in  Appendix  B.)   Univariate  ANOVA  revealed  that  method 
of  error  interruption  significantly  affected  total  time  and 
keypress  efficiency  on  both  tasks.   At  a  .05  significance 
level,  the  End  interruption  group  took  longer  to  complete 
each  task  and  was  less  efficient  at  the  keyboard  than  the 
Immediate  group.   These  results  are  not  independent,  however, 
due  to  the  high  negative  correlation  between  time  and  key- 
press efficiency.   Tables  3.2-c  and  3.2-d  contain  ANOVA  sum- 
mary tables  for  significant  results  on  each  task.   (Refer 
to  Appendix  D  for  summary  tables  of  all  ten  univariate  anal- 
yses. ) 

For  the  Permutations  task  the  direction  in  the  re- 
sults decisively  favors  the  Immediate  interruption  group. 
(The  only  result  on  this  task  favorable  to  the  End  group  was 
percent  error  keypresses,  but  the  difference  in  group  means 
(0.0  6)  was  negligible.)   On  the  States  task,  however,  direc- 
tion of  results  was  mixed.   The  End  group  had  fewer  percent 
error  keypresses  and  fewer  responses  in  error. 

Results  of  the  two-way  ANOVA  supported  those  of  the 
one-way  analyses  and  turned  up  no  interactions  between  error 
interruption  method  and  task.   It  turned  out,  however,  that 
task  made  a  difference  in  subject  performance.   Although 
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Table  3.2-a 
Summary  of  Performance  Data  for  States  Task 

Method  of  Error  Interruption 
Immediate  End 


Percent  Error  Keypresses 

Mean  2.55  1.9  9 

St.  Dev.  2.45  2.19 

* 

Total  Time  (seconds) 

Mean  234.0  300. 3 

St.  Dev.  94.8  147. 6 


Two  Consecutive  Re- 
sponses in  Error 

Mean  1.17  1.17 

St.  Dev.  1.46  1.65 


Number  of  Responses 
in  Error 

Mean  4.29  3.77 

St.  Dev.  3.11  3.17 

Keypress  Efficiency 
(percent) 

Mean  90.94  84.91 

St.  Dev.  8.06  13.11 


Based  on  sample  size  of  35  per  group. 
Significant  at  the  .05  level. 
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Table  3.2-b 
Summary  of  Performance  Data  for  Permutations  Task 

Method  of  Error  Interruption 

Immediate  End 

Percent  Error  Keypresses 

Mean  4.54  4.48 

St.  Dev.  2.21  3.4  5 

* 
Total  Time  (seconds) 

Mean  408.8  546.4 

St.  Dev.  174.9  232.4 


Two  Consecutive  Re- 
sponses in  Error 

Mean  1.09  2.77 

St.  Dev.  1.40  5.03 


Number  of  Responses 
in  Error 

Mean  4.4  6  4.83 

St.  Dev.  2.14  5.10 

* 

Keypress  Efficiency 

(percent) 

Mean  83.09  75.57 

St.  Dev.  9.50  18.34 


Based  on  sample  size  of  35  per  group, 
Significant  at  a  .05  level. 
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Table  3.2-c 
Analysis  of  Variance  Summary  Tables  for  States  Task 


Total 

Time  (seconds) 

Source  of 
Variance 

Sum  of 
Squares 

Degrees  of 
Freedom 

Mean 
Square 

F 
Ratio 

ME  I 
Error 

76778.730 
1076415.2 

1 
68 

76778.730 
15829.636 

4.850 

p  <  .05 


Keypress  Eff 

iciency  (per* 

cent) 

Source  of 
Variance 

Sum  of 
Squares 

Degrees  of 
Freedom 

Mean 
Square 

F 
Ratio 

ME  I 
Error 

636.014 
8290.629 

1 
68 

636.014 
121.921 

5.217 

p  <  .05 
i 

Method  of  Error  Interruption. 
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Table  3.2-d 
Analysis  of  Variance  Summary  Tables  for  Permutations  Task 


Total 

T 

ime  (seconds) 

Source  of 
Variance 

Sum  of 
Squares 

Degrees  of 
Freedom 

Mean 
Square 

F 
Ratio 

ME  I 
Error 

331437.00 
2960801.3 

1 
68 

331437.00 
43541.196 

7.612 

p  <  .05 


Keypress 

Eff 

iciency  (perc 

:ent) 

Source  of 
Variance 

Sum  of 
Squares 

Degrees  of 
Freedom 

Mean 
Square 

F 
Ratio 

ME  I 
Error 

988.129 
14937.314 

1 
68 

988.129 
219.666 

4.498 

p  <  .05 


Method  of  Error  Interruption. 
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effects  of  task  are  not  relevant  to  the  original  research 
questions,  it  is  interesting  to  note  that  the  Permutations 
task  took  longer  to  complete  and  caused  a  higher  percentage 
of  error  keypresses  and  lower  keypress  efficiency  than  did 
the  States  task.   (Level  of  significance  was  .05;  see  Appen- 
dix E  for  complete  summary  tables.) 

Multivariate  ANOVA  on  the  performance  measures  re- 
vealed significant  effects  of  error  interruption  method  on 
combined  performance  measures  in  each  task.   This  backs  up 
the  isolated  findings  of  univariate  analyses.   Discriminant 
function  weights  are  shown  in  Tables  3.2-e  and  3.2-f.   The 
standardized  weights  are  computed  from  the  raw  coefficients, 
taking  into  account  the  different  standard  deviations  of  the 
dependent  variables. 

The  signs  of  the  discriminant  weights  generally 
reflect  the  directions  in  performance  on  the  two  tasks, 
while  their  absolute  values  indicate  which  variables  are  most 
effective  in  separating  the  groups.   On  the  States  task, 
number  of  responses  in  error  and  keypress  efficiency  seem  to 
be  the  important  variables,  while  on  the  Permutations  task 
these  two  along  with  two  consecutive  responses  in  error  are 
singled  out.   It  might  seem  that  total  time,  which  was  signi- 
ficant in  univariate  tests,  has  vanished  from  importance, 
but  actually  it  was  negatively  correlated  with  keypress 
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Table  3.2-e 

Multivariate  ANOVA — States  Task 

Based  on  5  and  64  degrees  of  freedom,  the  F  ratio  was  cal- 
culated as  5.3861  (p  <  .05). 

Discriminant  Function  Coefficients 
Variable  Raw  Coefficient     Standardized 

Pet.  Error  Keypresses        -0.293085  -0.6915 

Total  Time  (seconds)  0.001590  0.2000 

Two  Consecutive  Re- 
sponses in  Error  -0.095394  -0.1508 

Number  of  Responses 

in  Error  -0.273706  -0.8727 

Keypress  Efficiency  (pet.)    -0.143781  -1.5876 


Table  3.2-f 

Multivariate  ANOVA — Permutations  Task 

Based  on  5  and  64  degrees  of  freedom,  the  F  ratio  was  cal- 
culated as  13.0327  (p  <  .05). 

Discriminant  Function  Coefficients 

Variable  Raw  Coefficient     Standardized 

Pet.  Error  Keypresses        -0.035850  -0.1054 

Total  Time  (seconds)  0.002440  0.5091 

Two  Consecutive  Re- 
sponses in  Error  0.766626  2.8728 

Number  of  Responses 

in  Error  -1.079513  -4.2855 

Keypress  Efficiency  (pet.)    -0.101125  -1.4988 
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efficiency,  so  its  effect  is  mostly  described  by  the  discri- 
minant weight  for  the  latter. 

The  discriminant  functions  indicate  that  method  of 
error  interruption  has  an  effect  on  a  linear  combination  of 
the  variables  with  high  weights.   This  effect  is  stronger 
than  that  on  some  of  the  measures  alone,  as  in  the  case  of 
number  of  responses  in  error  and  two  consecutive  responses 
in  error,  for  which  univariate  results  yield  no  significant 
differences. 

Multivariate  ANOVA  conducted  on  personality  traits 
produced  no  significant  differences — neither  on  error  inter- 
ruption method  or  on  performance  category.   Nor  were  there 
any  interactions  between  these  two  factors. 

Correlations  between  personality  traits  and  perform- 
ance scores  did,  however,  produce  some  significant  results. 
Four  correlation  matrices  were  produced  based  on  the  follow- 
ing breakdown : 

Immediate  interruption — States  task  (sample  size  =  31) 
Immediate  interruption — Permutations  task  (sample  size  =  31) 
End  interruption--States  task  (sample  size  =  30) 
End  interruption--Permutations  task  (sample  size  =  30) 
Sample  sizes  used  were  not  the  group  sample  sizes  due  to 
nine  subjects  who  either  could  not  take  the  16  PF  Test,  or 
refused  to  place  identification  on  their  answer  sheets. 
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Major  significant  correlations  are  reported  in 
Tables  3.2-g  and  3.2-h.   (See  Appendix  H  for  brief  descrip- 
tions of  personality  traits  involved.)   On  the  States  task, 
more  intelligent  subjects  in  both  groups  performed  better. 
Immediate  interruption  subjects  who  were  more  tense  and  End 
interruption  subjects  who  were  less  emotionally  stable  also 
did  better.   Possible  explanations  are  presented  in  the  next 
section. 

On  the  Permutations  task  only  shrewdness  and  intel- 
ligence remained  important  to  good  performance  in  the  Immed- 
iate and  End  groups,  respectively. 

In  all  correlation  results,  although  some  of  the 
coefficients  shown  in  Tables  3.2-g  and  3.2-h  are  not  signi- 
ficant at  a  «05  level,  direction  of  these  personality- 
performance  correlations  is  always  consistent. 

3.3   Discussion 

Analysis  of  the  performance  data  indicates  that  the 
Immediate  interruption  group  did  a  better  job  than  the  End 
group.   The  latter  group  was  significantly  slower  and  less 
efficient  (in  terms  of  keypresses)  on  each  task. 

There  are  several  probable  reasons  for  these  re- 
sults.  First,  the  instant  feedback  built  into  Immediate 
error  interruption  may  have  given  that  group  an  advantage  by 
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Table  3.2-g 

Major  Significant  Correlations  between  Personality 
Traits  and  Performance  Measures — States  Task 

Sample  Size  =  31 


_,      ,  .                     Performance  Measures 
Personality      

Factor  12         3        4 


Immediate  Error  Interruption 

B:   Intelligence  -0.36330*  -0.28256   -0.26581  -0.22648   0.32929 

Q.:  Tension       -0.39430*  -0.34104   -0 . 53568*-0 . 40673*  0.43559 
4 

End  Error  Interruption 

B:   Intelligence  -0.33980   -0.46204*  -0.22813  -0.25229   0.46954' 

C:   Emotional  *  *         * 

Stability      0.40539    0.31506    0.42508   0.43151  -0.31536 


Performance  Measures 

1.  Percent  Error  Keypresses 

2.  Total  Time  (seconds) 

3.  Two  Consecutive  Responses  in  Error 

4.  Number  of  Responses  in  Error 

5.  Keypress  Efficiency  (percent) 

* 

Significant  at  a  .05  level;  Critical  value  (from 

Ferguson  [3]),  with  29  degrees  of  freedom,  was  .355. 
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Table  3.2-h 

Major  Significant  Correlations  between  Personality 
Traits  and  Performance  Measures — 
Permutations  Task 

Sample  Size  =  30 


_,      ,  .  .                    Performance  Measures 
Personality      


Factor 


Immediate  Error  Interruption 


N:   Shrewdness    -0.44503*  -0.29648   -0.52688  -0.53060   0.44530 


End  Error  Interruption 
B:   Intelligence  -0.24363   -0.56538*  -0.38138  -0.42832   0.32757 


Performance  Measures 

1.  Percent  Error  Keypresses 

2.  Total  Time  (seconds) 

3.  Two  Consecutive  Responses  in  Error 

4.  Number  of  Responses  in  Error 

5.  Keypress  Efficiency  (percent) 

* 

Significant  at  a  .05  level;  critical  value  (from 

Ferguson  [3]),  with  28  degrees  of  freedom,  was  .361. 
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allowing  the  subjects  to  catch  their  errors  early  and  avoid 
making  the  same  type  of  mistake  more  than  once.   In  fact, 
several  End  interruption  subjects  made  one  consistent  error 
on  the  Permutations  task,  such  as  omitting  the  letter  '  e'  in 
each  response.   These  subjects  did  not  learn  of  their  mis- 
takes until  they  completed  their  responses,  and  thus  produced 
data  reflecting  a  very  poor  performance.   Immediate  interrup- 
tion subjects  under  a  similar  misconception  would  have  been 
made  aware  of  it  at  the  very  first  error.   Also,  in  terms  of 
keypresses,  error  correction  in  the  Immediate  group  required 
the  retyping  of  one  character  whereas  an  error  in  the  End 
group  required  that  the  whole  response  be  retyped  at  the 
task's  end.   This  could  partially  explain  the  lower  keypress 
efficiency  of  the  End  group. 

A  second  reason  for  the  better  performance  of  the 
Immediate  group  could  be  that  the  End  error  interruption 
method  allowed  for  more  individual  differences  among  subjects 
to  surface.   As  shown  in  Tables  3.2-a  and  3.2-b,  the  stan- 
dard deviations  of  scores  in  the  End  group  are  consistently 
higher  than  those  for  the  Immediate  group.   This  suggests 
that  the  rather  free  nature  of  End  interruption  causes  it  to 
be  quite  sensitive  to  individual  differences.   The  resulting 
rise  in  group  variability  may  have  washed  out  some  meaningful 
effects  of  the  different  types  of  error  interruption. 
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Considering  the  mixture  of  directions  in  performance 
measures  on  the  States  task,  it  is  quite  possible  that  the 
task  itself  is  the  key  to  determining  which  error  interrup- 
tion method  is  best.   The  Permutations  task,  the  more  diffi- 
cult and  mathematical  of  the  two,  may  have  been  ripe  for 
Immediate  interruption,  whereas  the  States  task,  more  related 
to  individual  experience,  produced  some  good  performance 
under  End  interruption. 

(As  mentioned  in  the  previous  section,  multivariate 
ANOVA  indicated  that  error  interruption  method  affected  a 
combination  of  certain  performance  measures.   However,  since 
no  meaningful  interpretation  was  found  for  these  combined  ef- 
fects it  is  unlikely  that  these  multivariate  findings  can  shed 
additional  light  on  the  present  discussion.) 

Correlations  of  personality  traits  with  performance 
measures  also  support  the  importance  of  task  as  a  factor.   On 
the  States  task,  Immediate  group  subjects  who  were  more  tense 
performed  better.   Possibly  the  instant  feedback  provided  by 
Immediate  interruption  helped  to  reassure  the  more  fretful 
subjects.   In  the  End  group,  lower  emotional  stability  led  to 
better  performance.   According  to  the  Handbook  for  the  16  PF 
[2] ,  high  tension  and  low  emotional  stability  are  sometimes 
difficult  to  distinguish,  so  possibly  the  same  basic  traits 
are  reflected  by  both  measures.   In  fact  for  the  Immediate 
group,  tension  and  emotional  stability  were  negatively 
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correlated.   Motivation  may  explain  some  of  this.   Tense  and 
excited  subjects  may  have  been  concerned  about  the  possibil- 
ity of  making  errors  and  therefore  may  have  done  better  than 
subjects  who  were  less  concerned  with  their  performance.   The 
Handbook  for  the  16  PF  [2]  points  out  that  ".  .  .in  simple 
performances  or  in  trial-and-error  performances,  where  high 
activity  level  is  important,  performance  has  been  found  posi- 
tively related  to  Q.  [Tension]."   The  "high  activity  level" 
might  be  the  result  of  an  excitable  subject  faced  with  a 
test-like  situation  at  a  computer  terminal. 

Fewer  significant  correlations  occurred  on  the 
Permutations  task.   For  the  Immediate  group,  more  shrewdness 
implied  better  performance,  while  in  the  End  group  intelli- 
gence was  the  dominant  factor.   Based  on  their  definitions 
these  two  factors  are  quite  similar,  so  once  again  the  same 
qualities  are  affecting  performance  in  both  error  interrup- 
tion groups.   It  is  dangerous,  of  course,  to  assume  that  the 
16  PF  Test  accurately  measures  intelligence — whole  tests  have 
been  designed  to  grapple  with  this  trait  alone.   It  does  seem 
reasonable,  however,  since  intelligence  correlations  remain 
significant  in  both  tasks,  to  claim  that  more  intelligent 
subjects  do  better  at  interactive  terminals  under  any  condi- 
tions. 

It  is  notable  that  many  significant  correlations  on 
the  States  task  disappear  on  the  Permutations  task.   Only 
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intelligence/shrewdness  seems  to  remain  important  in  the 
latter  task.   Listing  permutations  gave  the  subjects  con- 
siderably more  difficulty  than  did  listing  states  and  so  once 
again  the  tasks  seem  to  make  the  difference.   The  lack  of 
personality — performance  correlations  on  the  Permutations 
task  implies  that  in  more  difficult  tasks,  personality  fac- 
tors (with  the  exception  of  intelligence)  play  little  role 
in  subject  performance  at  the  terminal. 

Although  Immediate  error  interruption  was  favored 
in  this  study,  further  experimentation  in  this  area  would 
probably  be  useful.   For  example,  effects  of  error  interrup- 
tion on  programming  might  be  tackled  in  a  semester  long  study 
which  would  compare  two  classes  learning  to  program  through 
the  use  of  interactive  terminals. 

All  in  all,  studies  such  as  ours,  which  examine  a 
part  of  man-computer  interaction,  are  both  feasible  and  use- 
ful.  A  wide  range  of  data  can  be  collected  by  the  computer 
during  such  an  experiment,  and  the  system  can  provide  accurate 
control  of  many  variables.   The  university  is  an  excellent 
place  for  such  research  because  of  the  availability  of  sub- 
jects, equipment,  and  statistical  services. 
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4 .   SUMMARY 

If  a  student  makes  an  error  while  typing  material 
into  an  interactive  computer  terminal,  the  system  can  respond 
to  the  error  in  several  ways,  ranging  from  immediate  inter- 
ruption of  the  student's  work  to  delay  of  the  interruption 
until  the  student  decides  that  his  work  is  finished.   These 
two  methods  of  error  interruption — Immediate  and  End — were 
studied  to  determine  whether  the  method  used  affects  perform- 
ance.  It  was  hypothesized  that  Immediate  interruption  would 
damage  performance  by  jarring  the  student's  train  of  through. 
The  relationship  of  human  personality  to  performance  was  also 
examined. 

The  PLATO  IV  system  was  used  to  implement  the  major 
part  of  the  experiment,  with  subjects  chosen  from  basic  com- 
puter science  courses. 

The  experimental  procedure  consisted  of  two  phases: 
First,  subjects  were  given  the  16  PF  Test,  a  personality 
test  developed  at  the  University  of  Illinois  at  Urbana- 
Champaign,  which  measures  sixteen  basic  personality  traits. 
Next,  subjects  took  a  PLATO  lesson  which  asked  them  to  com- 
plete two  simple  tasks: 

1.  List  25  of  the  50  states. 

2.  List  20  permutations  of  'abcde'  such  that  the 
'c'  occurs  somewhere  before  the  'a'  and  the  'b' 
occurs  somewhere  before  the  *d'. 
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Half  the  subjects  experienced  Immediate  interruption  when 
making  an  error,  while  the  other  half  experienced  End  inter- 
ruption.  PLATO  experience  and  typing  skill  were  taken  into 
consideration  by  the  lesson  both  in  assigning  subjects  to 
treatment  groups  and  in  randomizing  the  order  in  which  the 
tasks  were  presented. 

Various  comprehensive  data  were  collected  on  each 
subject  during  the  PLATO  phase  of  the  procedure,  and  five 
dependent  variables  were  chosen  as  most  indicative  of  per- 
formance.  Each  subject  produced  two  scores  (one  for  each 
task)  on  each  of  the  following  variables: 

Percent  Error  Keypresses 

Total  Time  (seconds) 

Two  Consecutive  Responses  in  Error 

Number  of  Responses  in  Error 

Keypress  Efficiency  (percent) 

Significant  differences  appeared  in  the  multivariate 
analysis  of  variance  on  the  dependent  variables,  and  the 
discriminant  function  weights  generated  verify  results  on  the 
univariate  analyses.   Two  variables  showed  significant  dif- 
ferences in  simple  analysis  of  variance — total  time  and  key- 
press efficiency.   The  direction  in  both  of  these  favored 
the  Immediate  interruption  group,  i.e.,  they  took  significantly 
less  time  to  complete  each  task  and  were  significantly  more 
efficient  in  their  use  of  the  keyboard  (at  a  .05  level).   It 
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should  be  noted,  however,  that  these  results  were  not  indepen- 
dent, since  the  two  variables  were  highly  correlated  in  a 
negative  direction. 

Although  these  results  imply  that  the  Immediate 
interruption  group  performed  better,  the  direction  of  overall 
results  was  mixed.   For  example,  on  both  tasks,  the  End  group 
had  a  smaller  percentage  of  error  keypresses,  and  on  the 
States  task,  this  group  had  fewer  responses  in  error.   In 
general,  results  on  the  Permutations  task  favored  the  Immed- 
iate group,  while  results  on  the  States  task  were  mixed. 

The  multivariate  ANOVA  on  personality  scores  dis- 
played no  significant  differences,  neither  across  error  in- 
terruption method  nor  across  performance  categories.   Nor  did 
any  interactions  turn  up  in  this  analysis. 

There  were,  however,  significant  correlations  be- 
tween personality  measures  and  the  five  performance  measures 
mentioned  above.   Results  indicated  that  in  the  States  task 
tension  and  instability,  along  with  intelligence,  led  to 
better  performance.   Calmer,  more  stable  subjects  may  have 
been  less  motivated  to  do  well  than  more  excitable  subjects 
who,  when  faced  with  a  test-like  situation,  were  more  keyed 
up  and  alert.   On  the  Permutations  task,  however,  most  per- 
sonality relationships  disappeared.   Only  intelligence  re- 
mained important  to  performance,  as  it  had  also  been  in  the 
States  task.   Apparently,  as  the  task  becomes  more  difficult, 
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basic  intelligence  becomes  the  more  dominant  performance- 
affecting  trait. 

In  conclusion,  the  superior  performance  of  the 
Immediate  interruption  group  indicates  that  this  form  of 
interruption  has  no  adverse  affect  on  performance,  and  in 
fact  may  have  had  some  beneficial  effect.   In  addition,  per- 
sonality seems  to  affect  subject  performance  at  a  computer 
terminal  under  certain  conditions. 

Results  as  a  whole,  however,  imply  that  different 
tasks  and  different  personalities  require  different  methods 
of  error  interruption  in  order  to  maximize  effectiveness  at 
the  terminal.   Therefore,  further  studies  to  examine  the  ef- 
fects of  error  interruption  method  on  more  complicated  tasks 
such  as  programming  at  the  terminal  are  recommended. 
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Appendix  A 

RAW  DATA  COLLECTED  BY  PLATO  LESSON 

The  following  is  a  description  of  all  the  data 
collected  by  PLATO  during  the  experiment.   Of  the  data  items 
listed,  only  total  time  (scaled  to  seconds)  was  used  directly 
as  a  dependent  variable.   The  other  items  were  either  used 
to  derive  other  measures  or  were  biased  toward  one  of  the 
error  interruption  groups.   The  term  "initial  entry"  is  used 
to  describe  the  period  during  which  subjects  first  typed 
their  responses.   "Answer  correction"  denotes  the  period  dur- 
ing which  End  subjects  were  allowed  to  change  answers  after 
initial  entry.   "System  guided  correction"  refers  to  the 
period  after  answer  correction  during  which  End  subjects  were 
guided  through  corrections  by  the  system. 

Task  Data  (one  score  per  subject  per  task) 

See  Tables  A-l  and  A-2  for  summaries  of  results  on 
these  items. 

Total  Error  Keypresses: 

Total  number  of  error  keypresses  made  on  the 
task  (in  End  interruption,  error  checking  in 
a  response  was  suspended  at  the  occurrence  of 
an  error,  until  that  error  was  corrected  or 
the  response  completed. 
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Table  A-l 


Summary  of  Raw  Task  Data — States  Task 


Method  of  Error  Interruption 


Immediate  End 


Total  Error  Keypresses 

Mean  7.00  6.37 

St.  Dev.  7.38  7.76 

Min.*  0  0 

Max.*  35  37 

Total  Time  (sec.) 

Mean  234.0  300.3 

St.  Dev.  94.8  147.6 

Min.  126.7  119.3 

Max.  525.1  738.2 

Total  Erase  Presses 

Mean  9.31  10.03 

St.  Dev.  9.65  10.27 

Min.  0  0 

Max.  39  43 

Total  Erase  Operations 

Mean  7.69  4.51 

St.  Dev.  7.93  4.03 

Min.  0  0 

Max.  37  16 

Total  Keypresses 

Mean  255.94  281.83 

St.  Dev.  23.86  54.90 

Min.  217  216 

Max.  319  44  8 
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Table  A-l  (continued) 


Method  of  Error  Interruption 


Immediate  End 


Number  of  Fooling-Around  Times 

Mean  0.20  0.00 

St.  Dev.  0.71  0.00 

Min  0  0 

Max.  4  0 

Total  Editing  Time  (sec.) 

Mean  0.0  44.5 

St.  Dev.  0.0  74.8 

Min.  0.0  0.0 

Max.  0.0  318.3 

Number  of  Times  Edit  Occurred 

Mean  0.0  0.69 

St.  Dev.  0.0  1.17 

Min.  0  0 

Max.  0  5 

* 
Min.  is  minimum  score;  Max.  is  maximum  score. 
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Table  A-2 


Summary  of  Raw  Task  Data — Permutations  Task 


Method  of  Error  Interruption 
Immediate  End 


Total  Error  Keypresses 

Mean  7.00  9.31 

St.  Dev.  4.38  9.56 

Min.*  2  0 

Max.*  2  0  3  3 

Total  Time  (sec.) 

Mean  408.8  546.4 

St.  Dev.  174.9  232.4 

Min.  168.4  194.7 

Max.  911.4  1212.6 

Total  Erase  Presses 

Mean  10.91  6.51 

St.  Dev.  8.74  6.58 

Min .  3  0 

Max.  46  33 

Total  Erase  Operations 

Mean  8.34  4.00 

St.  Dev.  5.36  4.13 

Min.  3  0 

Max.  27  20 

Total  Keypresses 

Mean  146.77  173.91 

St.  Dev.  21.19  59.89 

Min.  127  122 

Max.  2  36  36  7 
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Table  A-2  (continued) 


Method  of  Error  Interruption 
Immediate  End 


Number  of  Fooling-Around  Times 

Mean                          0.14  0.00 

St.  Dev.                       0.35  0.00 

Min.                           0  0 

Max.                           1  0 

Total  Editing  Time  (sec.) 

Mean                          0.0  69.7 

St.  Dev.                       0.0  147.9 

Min.                           0.0  0.0 

Max.                           0.0  708.6 

Number  of  Times  Edit  Occurred 

Mean                          0.0  2.77 

St.  Dev.                       0.0  4.09 

Min.                           0  0 

Max.                           0  20 

* 

Min.  is  minimum  score;  Max.  is  maximum  score. 
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Total  Time  (tenths  of  seconds) : 

Time  from  presentation  of  task  layout  screen 
to  successful  completion  of  task. 

Total  Erase  Presses: 

Total  number  of  keypresses  made  to  erase  char- 
acters . 

Total  Erase  Operations: 

One  erase  operation  equals  N  consecutive  erase 
presses  (N  being  an  integer  greater  than  zero)  . 

Total  Keypresses: 

Total  number  of  keypresses  made  during  task, 
excluding  presses  of  special  function  keys 
which  were  not  active. 

Number  of  Fooling-Around  Times: 

Fooling-around  was  defined  as  making  three  con- 
secutive error  keypresses  at  the  same  position 
in  the  response.   The  purpose  of  this  measure 
was  to  determine  whether  Immediate  interruption 
subjects  were  using  trial  and  error  to  figure 
out  which  characters  the  lesson  would  accept. 

Total  Editing  Time  (tenths  of  seconds) : 

(End  group  only)  total  time  from  completion 
of  responses  to  successful  completion  of  task. 

Number  of  Edit  Occurrences: 

(End  group  only)  number  of  times  subject 
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changed  a  previously  typed  response  during 
initial  entry  and  before  system  guided  correc- 
tion. 

Bit  Arrays  (one  bit  per  response  per  subject  per  task) 

(Note:   The  first  six  arrays  apply  only  to  the 
initial  entry  period. ) 

Error  Correction: 

For  the  Immediate  group — occurrence  of  an 
interruption  by  the  lesson. 

For  the  End  group — erasure  of  an  erroneous 
character. 
Multiple  Error  Correction: 

More  than  one  occurrence  of  an  error  correction 

as  defined  above. 
Unnecessary  Erasure: 

Erasure  of  a  correct  character. 
Multiple  Unnecessary  Erasures: 

More  than  one  occurrence  of  an  unnecessary 

erasure. 
Error  Occurrence: 

Occurrence  of  an  error  in  the  response. 
Interruption  of  a  Previous  Response: 

(End  group  only)  interruption  of  the  current 

response  to  change  a  previously  typed  response. 
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Such  interruptions  could  only  occur  at  the  very- 
beginning  of  the  current  response. 

Wrong  at  Finish: 

(End  group  only)  response  was  wrong  when  sub- 
ject signalled  completion  of  task. 

Edit  Occurrence: 

(End  group  only)  response  was  later  changed. 

Edit  Error  Occurrence: 

(End  group  only)  error  occurred  while  response 
was  being  changed. 


Other  Multiple  Data 


Response  Times  (tenths  of  seconds) : 

Times  for  each  response  of  each  subject  on 
each  task.   Does  not  include  changing  of 
answers  by  End  group  after  responses  were 
typed. 

Final  Answers: 

Complete  list  of  answers  arrived  at  by  each 
subject  on  each  task. 
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Appendix  B 

MEASURES  DERIVED  FROM  RAW  DATA 

Various  measures  were  derived  from  the  raw  data 
given  in  Appendix  A.   Summaries  of  the  derived  task  data 
are  given  in  Tables  B-l  and  B-2.   Definitions  of  the  derived 
data  follow: 

Interruption  Bit  Array  (one  bit  per  response  per  subject  per 

task) 

Denotes  whether  the  response  was  interrupted  in 
any  way.   Computed  as  the  logical  OR  of  elements  in  the 
error  correction/  unnecessary  erasure,  and  interruption  of 
previous  response  arrays. 

Task  Data  (one  score  per  subject  per  task) 

Number  of  Responses  Interrupted: 

Computed  from  interruption  bit  array. 
Number  of  Responses  in  Error: 

Computed  from  error  bit  array. 
Two  Consecutive  Responses  Interrupted: 

Computed  from  interruption  bit  array. 
Two  Consecutive  Responses  in  Error: 

Computed  from  error  bit  array. 
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Table  B-l 


Summary  of  Derived  Task  Data — States  Task 


Method  of  Error  Interruption 
Immediate  End 


Number  of  Responses  Interrupted 

Mean  4.40  3.43 

St.  Dev.  3.12  2.32 

Min.*  0  0 

Max.*  .12  9 

Two  Consecutive  Responses  Interrupted 

Mean  1.2  3  0.80 

St.  Dev.  1.48  1.04 

Min.  0  0 

Max.  5  4 

Number  of  Responses  in  Error 

Mean  4.29  3.77 

St.  Dev.  3.11  3.17 

Min.  0  0 

Max.  12  12 

Two  Consecutive  Responses  in  Error 

Mean  1.17  1.17 

St.  Dev.  1.46  1.65 

Min.  0  0 

Max.  5  6 

Background  Time  (sec. ) 

Mean  7.5  7.9 

St.  Dev.  2.3  2.1 

Min  4.5  3.4 

Max.  13.8  11.9 
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Table  B-l  (continued) 


Method  of  Error  Interruption 


Immediate 


End 


Damage  Time  (sec. ) 

Mean 
St.  Dev. 

Min. 
Max. 


10.  5 
4.8 
3.6 

24.5 


10.7 
5.8 
2.9 

32.5 


Interruption  Damage 

Mean 
St.  Dev. 
Min. 
Max. 


39.8 

48.5 

-37.9 

217.8 


35.8 

73.2 

-40.2 

322.1 


Number  of  Unnecessary  Erasures 


Mean 
St.  Dev. 
Min. 
Max. 


1.11 
1.21 
0 
4 


3.14 
2.23 
0 
9 


Perfect  Number  of  Keypresses 

Mean  2  31.09 

St.  Dev.  11.33 

Min.  209 

Max.  260 


232.54 
11.84 
211 
257 


Keypress  Efficiency  (percent) 


Mean 
St.  Dev. 

Min. 
Max. 


90.94 
8.06 
70 
100 


84.91 
13.11 
55 
100 


Percent  Error  Keypresses  (NR) 


Mean 
St.  Dev. 
Min. 
Max. 


28.00 
29.53 
0 
140 


25.49 
31.03 
0 
148 


59 


Table  B-l  (continued) 


Method  of  Error  Interruption 


Immediate  End 


Percent  Error  Keypresses  (PNK) 

Mean                          3.07  2.70 

St.  Dev.                       3.32  3.21 

Min.                          0.00  0.00 

Max.                         16.05  14.85 

Percent  Error  Keypresses  (NKP) 

Mean                          2.55  1.99 

St.  Dev.                       2.45  2.19 

Min.                          0.00  0.00 

Max.                         11.62  11.11 

* 

Min.  is  minimum  score;  Max.  is  maximum  score. 
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Table  B-2 


Summary  of  Derived  Task  Data — Permutations  Task 


Method  of  Error  Interruption 


Immediate  End 


Number  of  Responses  Interrupted 

Mean  5.00  3.17 

St.  Dev.  2.23  2.16 

Min.*  1  0 

Max.*  12  11 

Two  Consecutive  Responses  Interrupted 

Mean  1.29  0.69 

St.  Dev.  1.58  1.39 

Min.  0  0 

Max.  6  7 

Number  of  Responses  in  Error 

Mean  4.46  4.83 

St.  Dev.  2.14  5.10 

Min.  1  0 

Max.  11  20 

Two  Consecutive  Responses  in  Error 

Mean  1.09  2.77 

St.  Dev.  1.40  5.03 

Min.  0  0 

Max.  5  19 

Background  Time  (sec. ) 

Mean  14.0  15.0 

St.  Dev.  6.4  4.6 

Min.  5.1  6.3 

Max.  41.7  25.8 
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Table  B-2  (continued) 


Method  of  Error  Interruption 


Immediate 


End 


Damage  Time  (sec. ) 

Mean 
St.  Dev. 
Min. 
Max. 


21.0 

11.0 

6.6 

52.6 


25.2 

18.0 

3.7 

74.9 


Interruption  Damage 

Mean 
St.  Dev. 
Min. 
Max. 


64.5 

84.4 

-61.2 

329.6 


71.  3 
110.3 
-60.6 
406.1 


Number  of  Unnecessary  Erasures 

Mean  2 .  14 

St.  Dev.  2.07 

Min.  0 

Max.  9 


2.23 
1.76 
0 
9 


Perfect  Number  of  Keypresses 

Mean  120.00 

St.  Dev.  0.00 

Min.  120 

Max.  120 


121.00 
0.00 
121 
121 


Keypress  Efficiency  (percent) 

Mean  83.09 

St.  Dev.  9.50 

Min.  51 

Max.  94 

Percent  Error  Keypresses  (NR) 


Mean 
St.  Dev, 
Min. 
Max. 


35.00 

21.91 

10.00 

100.00 


75.57 
18.34 
37 
99 


46.57 

47.78 

0.00 

165.00 
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Table  B-2  (continued) 


Method  of  Error  Interruption 


Immediate  End 


Percent  Error  Keypresses  (PNK) 

Mean                          5.83  7.6  9 

St.  Dev.                       3.65  7.90 

Min.                           1.66  0.00 

Max.                          16.66  27.27 

Percent  Error  Keypresses  (NKP) 

Mean                          4.54  4.4  8 

St.  Dev.                       2.21  3.45 

Min.                           1.48  0.00 

Max.                          10.61  11.95 

* 

Min.  is  minimum  score;  Max.  is  maximum  score. 
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Background  Time  (tenths  of  seconds) : 

Average  of  all  times  for  responses  which  were 
the  third  of  three  consecutive  uninterrupted 
responses  (excepting  the  first  three  re- 
sponses) .   This  represents  an  average  response 
time  during  smooth  response  flow,  when  no  re- 
cent interruptions  have  occurred.   The  first 
three  responses  were  not  considered  part  of 
the  normal  flow.   (The  counts  used  to  compute 
these  averages  were  also  collected.) 

Damage  Time  (tenths  of  seconds) : 

Average  of  all  times  for  responses  (excepting 
the  first  three)  which  follow  interrupted 
responses.   This  measures  the  gross  effect  an 
interruption  has  on  the  following  response. 
(Counts  used  to  compute  averages  were  also 
collected. ) 

Interruption  Damage: 

Computed  as 

inn  *  Dama9e  Time  -  Background  Time % 
Background  Time 

measures  the  net  damage  to  speed  caused  by 
interruptions . 
Number  of  Unnecessary  Erasures: 

Computed  from  unnecessary  erasure  bit  array. 
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Perfect  Number  of  Keypresses: 

Number  of  keypresses  necessary  to  produce 
subject ' s  final  task  answers  without  making 
any  errors . 

Keypress  Efficiency  (percent) : 

Calculated  as 

inn  *  Perfect  Number  of  Keypresses 
Total  Number  of  Keypresses 

Percent  Error  Keypresses  (NR) : 

Percent  error  keypresses  based  on  number  of 

responses;  calculated  as 

inn  *  T°tal  Error  Keypresses 
Number  of  Responses 

Percent  Error  Keypresses  (PNK) : 

Percent  of  error  keypresses  based  on  perfect 

number  of  keypresses;  calculated  as 
inn  *     Total  Error  Keypresses 


Perfect  number  of  Keypresses* 
Percent  Error  Keypresses  (NKP) : 

Percent  of  error  keypresses  based  on  number  of 

keypresses;  calculated  as 

inn  *   Total  Error  Keypresses 

Total  number  of  Keypresses 

(This  measure  was  used  as  a  dependent  variable.) 
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Appendix  C 

T-TEST  RESULTS 

T-Tests  were  performed  to  determine  whether  group 
means  on  typing  test  time,  typing  test  errors,  and  number  of 
years  of  programming  were  significant.   Results  of  these 
tests  are  shown  below.   The  critical  value  of  T  at  a  .05 
level  of  significance  for  the  two-tailed  test  is  1.6676, 

taken  from  Ferguson  [3].   Sample  size  for  each  group  was  35, 

degrees  of  freedom  was  68. 

i 

Typing  Test  Time  (.1  sec.) 


Group 


Immediate  End 


Mean  941.5  923.7 

St.  Dev.  277.5  266.1 


Value  of  T  was  0.001  (not  significant), 
p  <  .  05 

Typing  Test  Errors 


Group 


Immediate  End 


Mean  2.46  2.20 

St.  Dev.  1.99  1.67 


Value  of  T  was  0.318  (not  significant) 
p  <  .  05 


Number  of  Years  of  Programming 
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Group 


Immediate  End 


Mean  0.77  0.83 

St.  Dev.  1.20  1.52 


Value  of  T  was  0.183  (not  significant), 
p  <  .05 
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Appendix  D 

ONE-WAY  UNIVARIATE  ANOVA  SUMMARY  TABLES 

Results  of  ANOVA  on  each  performance  measure  for 
each  task  appear  on  the  following  pages.   States  task  re- 
sults are  given  in  the  first  five  tables,  followed  by 
Permutations  task  results  in  the  last  five.   (MEI  stands  for 
Method  of  Error  Interruption.) 
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Analysis  of  Variance  Summary  Tables 
Percent  Error  Keypresses — States 


Source  of  Sum  of 

Variance  Squares  DF  Mean  Square   F  Ratio   Probability 

MEI  5.601  1        5.601     1.006       0.319 

Error  378.577  68        5.567 


Total  Time  (seconds) — States 


Source  of    Sum  of 

Variance     Squares  DF  Mean  Square   F  Ratio   Probability 

MEI          76778.730  1  76778.730     4.850       0.031 

Error      1076415.200  68  15829.636 

Two  Consecutive  Responses  in  Error—States 

Source  of    Sum  of 

Variance     Squares  DF  Mean  Square   F  Ratio   Probability 

MEI              0.000  1  0.000     0.000       1.000 

Error          169.943  68  2.499 


Number  of  Responses  in  Error — States 


Source  of  Sum  of 

Variance  Squares  DF  Mean  Square   F  Ratio   Probability 

MEI  4.629  1        4.629     0.455       0.502 

Error  691.314  68       10.166 
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Analysis  of  Variance  Summary  Tables  (continued) 
Keypress  Efficiency  (percent) — States 

Source  of   Sum  of 

Variance     Square       DF   Mean  Square   F  Ratio   Probability 

MEI  636.014     1      636.014     5.217       0.026 

Error         8290.629    68      121.921 

Percent  Error  Keypresses — Permutations 

Source  of   Sum  of 

Variance     Square       DF   Mean  Square   F  Ratio   Probability 

MEI  0.061     1        0.061     0.007       0.934 

Error  587.777    68        8.644 

Total  Time  (seconds) — Permutations 

Source  of    Sum  of 

Variance     Square       DF   Mean  Square   F  Ratio   Probability 

MEI         331437.000     1   331437.000     7.612       0.007 
Error      2960801.300    68    43541.196 

Two  Consecutive  Responses  in  Error — Permutations 

Source  of   Sum  of 

Variance     Square       DF   Mean  Square   F  Ratio   Probability 

MEI  49.729     1       49.729     3.541       0.064 

Error  954.914    68       14.043 
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Analysis  of  Variance  Summary  Tables  (continued) 
Number  of  Responses  in  Error — Permutations 

Source  of    Sum  of 

Variance     Square       DF   Mean  Square   F  Ratio   Probability 

MEI  2.414     1        2.414     0.153       0.697 

Error         1071.657    68       15.760 

Keypress  Efficiency  (percent)- — Permutations 

Source  of    Sum  of 

Variance    Square      DF   Mean  Square   F  Ratio   Probability 

MEI  988.129     1      988.129     4.498       0.038 

Error        14937.314    68      219.666 
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Appendix  E 

TWO-WAY  UNIVARIATE  ANOVA  SUMMARY  TABLES 

Results  of  two-way  analyses  are  shown  in  the  fol- 
lowing pages.   The  summary  table  is  given  for  each  performance 
measure.   The  Geisser-Greenhouse  Conservative  F  Test  shown 
with  each  summary  table  is  a  check  for  homogeneity  of  co- 
variance  in  the  data.   The  test  made  no  difference  to  the 
results  shown  here. 
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Appendix  F 
CORRELATION  MATRICES 

Four  complete  correlation  matrices  of  performance 

measures  and  personality  traits  are  given  here: 
Immediate  interruption — States  task 
Immediate  interruption — Permutations  task 
End  interruption — States  task 
End  interruption — Permutations  task 
The  variables  are  numbered  as  follows: 

Variable  Number  Meaning 

(one  through  five  are  performance  measures) 

1  Percent  Error  Keypresses 

2  Total  Time  (seconds) 

3  Two  Consecutive  Responses  in  Error 

4  Number  of  Responses  in  Error 

5  Keypress  Efficiency  (percent) 

* 
(six  through  twenty-one  are  the  16  PF  traits) 

6  Reserved  vs.  Outgoing 

7  Less  Intelligent  vs .  More  Intelligent 

8  Affected  by  Feelings  vs.  Emotionally 

Stable 

9  Humble  vs.  Assertive 


* 
More  detailed  descriptions  of  those  traits  which  were 

important  are  given  in  Appendix  H.   This  list  is  taken  from 

the  Manual  for  the  16  PF.  [ 6l 
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Variable  Number  (continued)            Meaning  (continued) 

10  Sober  vs.  Happy-go-Lucky 

11  Expedient  vs.  Conscientious 

12  Shy  vs.  Venturesome 

13  Tough  Minded  vs.  Tender  Minded 

14  Trusting  vs.  Suspicious 

15  Practical  vs.  Imaginative 

16  Forthright  vs.  Shrewd 

17  Placid  vs.  Apprehensive 

18  Conservative  vs.  Experimenting 

19  Group  Dependent  vs.  Self  Suf- 

ficient 

20  Undisciplined  Self  Conflict  vs 

Controlled 

21  Relaxed  vs.  Tense 
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Appendix  G 

CLASSIFICATION  OF  PERFORMANCE  FOR  MULTIVARIATE  ANOVA 

In  order  to  use  performance  as  a  factor  in  the  two- 
way  multivariate  ANOVA  on  personality  data,  subject  perform- 
ance was  classified  into  four  categories  ranging  from  excel- 
lent to  poor.   A  performance  value  was  calculated  for  each 
subject  as  follows: 

2 

Performance  value  =  j      total  time^  +  F,  * 

t=l  fc    fc 

total  errors 

where  t  represents  task  number  and 

grand  total  time  mean 


F.  = 


t   grand  total  error  mean 

The  resulting  performance  values  were  then  sorted, 
the  smaller  values  representing  better  performance.   Subjects 

were  assigned  to  performance  categories  as  follows: 

Position  of  Value  in  List  Category 

Lowest  quarter  Excellent 

Next  to  lowest  quarter  Good 

Second  highest  quarter  Fair 

Highest  quarter  Poor 
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Appendix  H 

PERSONALITY  TRAITS  MEASURED  BY  THE  16  PF  TEST 

The  following  list  gives  brief  descriptions  of 
some  of  the  sixteen  primary  personality  factors  measured  by 
the  16  PF  Test.   The  list,  abstracted  from  the  Manual  for 
the  16  PF  [6]/  gives  those  factors  for  which  significant 
results  appeared. 

Factor  B 


Less  Intelligent,  Concrete- 
thinking  (Lower  scholastic 
mental  capacity) 

The  person  scoring  low  on 
Factor  B  tends  to  be  slow  to 
learn  and  grasp,  dull,  given 
to  concrete  and  literal  in- 
terpretation.  This  dullness 
may  be  simply  a  reflection 
of  low  intelligence,  or  it 
may  represent  poor  function- 
ing due  to  psychopathology. 


vs.  More  Intelligent,  Abstract- 
thinking,  Bright  (Higher 
scholastic  mental  capacity) 

The  person  who  scores  high 
on  Factor  B  tends  to  be  quick 
to  grasp  ideas,  a  fast  learner, 
intelligent.   There  is  some 
correlation  with  level  of 
culture,  and  some  with  alert- 
ness.  High  scores  contrain- 
dicate  deterioration  of  men- 
tal functions  in  pathological 
conditions. 


Factor  C 


Affected  by  Feelings ,  Emo- 
tionally Less  Stable 
Easily  Upset  (Lower  ego 
strength) 

The  person  who  scores  low 
on  Factor  C  tends  to  be  low 
in  frustration  tolerance  for 
unsatisfactory  conditions, 


vs.  Emotionally  Stable,  Faces 

Reality,  Calm,  Mature  (Higher 
ego  strength) 


The  person  who  scores  high 
on  Factor  C  tends  to  be  emo- 
tionally mature,  stable,  re- 
alistic about  life,  unruffled, 
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Factor  C  (continued) 


Affected  by  Feelings,  Emo- 
tionally Less  Stable 
Easily  Upset  (Lower  ego 
strength) 

changeable  and  plastic, 
evading  necessary  reality 
demands,  neurotically  fa- 
tigued, fretful,  easily  emo- 
tional and  annoyed,  active 
in  dissatisfaction,  having 
neurotic  symptoms  (phobias, 
sleep  disturbances,  psycho- 
somatic complaints,  etc.). 
Low  Factor  C  score  is  com- 
mon to  almost  all  forms  of 
neurotic  and  some  psychotic 
disorders. 


vs#  Emotionally  Stable,  Faces 

Reality,  Calm,  Mature  (Higher 
ego  strength) 


possessing  ego  strength, 
better  able  to  maintain 
solid  group  morale.   Some- 
times he  may  be  a  person 
making  a  resigned  adjust- 
ment to  unsolved  emotional 
problems. 


Factor  N 


Forthright,  Natural,  Art- 
less, Sentimental  (Art- 
lessness) 


vs.  Shrewd,  Calculating,  Wordly, 
Penetrating  (Shrewdness) 


The  person  who  scores  low 
on  Factor  N  tends  to  be  un- 
sophisticated, sentimental, 
and  simple.   He  is  some- 
times crude  and  awkward, 
but  easily  pleased  and  con- 
tent with  what  comes,  and 
is  natural  and  spontaneous. 


The  person 
on  Factor  N 
ished,  exper 
shrewd.   He 
headed  and  a 
has  an  intel 
sentimental 
uations ,  an 
to  cynicism. 


who  scores  high 
tends  to  be  pol- 
ienced,  worldly, 
is  often  hard- 
nalytical.   He 
lectual,  un- 
approach  to  sit- 
approach  akin 


Factor  Q 
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Relaxed,  Tranquil,  Torpid, 
Unfrustrated  (Low  ergic 
tension) 

The  person  who  scores  low 
on  Factor  Q4  tends  to  be 
sedate,  relaxed,  composed, 
and  satisfied  (not  frus- 
trated) .   In  some  situa- 
tions, his  oversatisfac- 
tion  can  lead  to  laziness 
and  low  performance,  in  the 
sense  that  low  motivation 
produces  little  trial  and 
error.   Conversely,  high 
tension  level  may  disrupt 
school  and  work  performance. 


vs.  Tense,  Frustrated,  Driven, 
Overwrought  (High  ergic 
tension) 

The  person  who  scores  high 
on  Factor  Q4  tends  to  be 
tense,  excitable,  restless, 
fretful,  impatient.   He  is 
often  fatigued,  but  unable 
to  remain  inactive.   In 
groups  he  takes  a  poor  view 
of  the  degree  of  unity, 
orderliness,  and  leadership. 
His  frustration  represents 
an  excess  of  stimulated,  but 
undischarged,  drive. 
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